{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "112cd03c1b2c85d",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 基于MindNLP在IAM手写数据集上微调TrOCR模型\n",
    "在本笔记本中，我们将在 IAM 手写数据库（一组手写文本的注释图像）上微调预训练的 TrOCR 模型。\n",
    "\n",
    "我们将使用新的 VisionEncoderDecoderModel 类来完成此任务，该类可用于将任何图像 Transformer 编码器（如 ViT、BEiT）与任何文本 Transformer 解码器（如 BERT、RoBERTa、GPT-2）组合。TrOCR 就是其中一个例子，因为它具有编码器-解码器架构，编码器的权重从预训练的 BEiT 初始化，解码器的权重从预训练的 RoBERTa 初始化。交叉注意层的权重是随机初始化的，之后作者在数百万张（部分合成的）手写文本注释图像上进一步预训练了该模型。\n",
    "\n",
    "该图很好地概述了模型（来自原始论文）：\n",
    "![Schermafbeelding 2021-10-26 om 16.09.25.png]()\n",
    "我们将使用原生 PyTorch 微调模型。\n",
    "\n",
    "设置环境\n",
    "\n",
    "首先，让我们安装所需的库：\n",
    "\n",
    "MindNLP（用于 TrOCR 模型）\n",
    "Datasets 和 Jiwer（用于评估指标）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4735fc4cb874bd67",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "!pip install git+https://gitee.com/mindspore-lab/mindnlp.git\n",
    "!pip install -q datasets jiwer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f237e5d8d9368af",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 准备数据\n",
    "我们首先下载数据。在这里，我只使用 IAM 测试集，因为这是 TrOCR 作者在 unilm 存储库中发布的。可以从这个页面下载。[https://github.com/microsoft/unilm/tree/master/trocr]\n",
    "\n",
    "让我们创建一个常规的 PyTorch 数据集。我们首先创建一个包含 2 列的 Pandas 数据帧。每一行由图像的文件名和相应的文本组成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "64243f60c0a21e78",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:15:12.569165600Z",
     "start_time": "2025-01-09T08:15:10.418374700Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>file_name</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>c04-110-00.jpg</td>\n",
       "      <td>Become a success with a disc and hey presto ! ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>c04-110-01.jpg</td>\n",
       "      <td>assuredness \" Bella Bella Marie \" ( Parlophone...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c04-110-02.jpg</td>\n",
       "      <td>I don't think he will storm the charts with th...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>c04-110-03.jpg</td>\n",
       "      <td>CHRIS CHARLES , 39 , who lives in Stockton-on-...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>c04-116-00.jpg</td>\n",
       "      <td>He is also a director of a couple of garages ....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        file_name                                               text\n",
       "0  c04-110-00.jpg  Become a success with a disc and hey presto ! ...\n",
       "1  c04-110-01.jpg  assuredness \" Bella Bella Marie \" ( Parlophone...\n",
       "2  c04-110-02.jpg  I don't think he will storm the charts with th...\n",
       "3  c04-110-03.jpg  CHRIS CHARLES , 39 , who lives in Stockton-on-...\n",
       "4  c04-116-00.jpg  He is also a director of a couple of garages ...."
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_fwf('./IAM/gt_test.txt', header=None)\n",
    "df.rename(columns={0: \"file_name\", 1: \"text\"}, inplace=True)\n",
    "del df[2]\n",
    "# some file names end with jp instead of jpg, let's fix this\n",
    "df['file_name'] = df['file_name'].apply(lambda x: x + 'g' if x.endswith('jp') else x)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35140cf70056daf9",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "我们使用 sklearn 的 train_test_split 函数将数据分成训练集和测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c69c51feb9586a34",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:15:27.557570800Z",
     "start_time": "2025-01-09T08:15:22.234824200Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "train_df, test_df = train_test_split(df, test_size=0.2)\n",
    "# we reset the indices to start from zero\n",
    "train_df.reset_index(drop=True, inplace=True)\n",
    "test_df.reset_index(drop=True, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c45ff3dcc72c02b2",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "我们将使用 `TrOCRProcessor` 为模型准备数据。`TrOCRProcessor` 实际上只是 `ViTFeatureExtractor`（可用于调整大小和标准化图像）和 `RobertaTokenizer`（可用于将文本编码和解码为 `input_ids`）的包装器。\n",
    "\n",
    "数据集的每个元素应返回两样东西：\n",
    "\n",
    "* `pixel_values`，用作模型的输入。\n",
    "* `labels`，是图像中相应文本的 `input_ids`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a8e73c388c4c5fae",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:15:35.431540900Z",
     "start_time": "2025-01-09T08:15:27.564640700Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from mindspore import Tensor\n",
    "from mindspore.dataset import Dataset\n",
    "from PIL import Image\n",
    "\n",
    "class IAMDataset():\n",
    "    def __init__(self, root_dir, df, processor, max_target_length=128):\n",
    "        super(IAMDataset).__init__()\n",
    "        self.root_dir = root_dir\n",
    "        self.df = df\n",
    "        self.processor = processor\n",
    "        self.max_target_length = max_target_length\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.df)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        # get file name + text\n",
    "        file_name = self.df['file_name'][idx]\n",
    "        text = self.df['text'][idx]\n",
    "        # prepare image (i.e. resize + normalize)\n",
    "        image = Image.open(self.root_dir + '/' + file_name).convert(\"RGB\")\n",
    "        pixel_values = self.processor(image, return_tensors=\"np\").pixel_values\n",
    "        # add labels (input_ids) by encoding the text\n",
    "        labels = self.processor.tokenizer(text,\n",
    "                                          padding=\"max_length\",\n",
    "                                          max_length=self.max_target_length).input_ids\n",
    "        # important: make sure that PAD tokens are ignored by the loss function\n",
    "        labels = [label if label != self.processor.tokenizer.pad_token_id else -100 for label in labels]\n",
    "\n",
    "        # encoding = {: , : }\n",
    "        return Tensor(pixel_values.squeeze()), Tensor(labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7aa4e1ab0a2812a9",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:16:13.646522400Z",
     "start_time": "2025-01-09T08:15:35.433609200Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\virgo\\.conda\\envs\\mindspore\\lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "Building prefix dict from the default dictionary ...\n",
      "Loading model from cache C:\\Users\\virgo\\AppData\\Local\\Temp\\jieba.cache\n",
      "Loading model cost 1.358 seconds.\n",
      "Prefix dict has been built successfully.\n",
      "100%|██████████| 224/224 [00:00<00:00, 224kB/s]\n",
      "1.09kB [00:00, 1.07MB/s]                 \n",
      "878kB [00:13, 66.4kB/s] \n",
      "446kB [00:00, 477kB/s] \n",
      "772B [00:00, 773kB/s]                    \n",
      "C:\\Users\\virgo\\Documents\\GitHub\\mindnlp\\mindnlp\\transformers\\tokenization_utils_base.py:1526: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from mindnlp.transformers.models.trocr import TrOCRProcessor\n",
    "\n",
    "processor = TrOCRProcessor.from_pretrained(\"microsoft/trocr-base-handwritten\")\n",
    "train_dataset = IAMDataset(root_dir='./IAM/image',\n",
    "                           df=train_df,\n",
    "                           processor=processor)\n",
    "eval_dataset = IAMDataset(root_dir='./IAM/image',\n",
    "                          df=test_df,\n",
    "                          processor=processor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "70649eb1d48f836b",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:16:13.686308500Z",
     "start_time": "2025-01-09T08:16:13.644522400Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of training examples: 2332\n",
      "Number of validation examples: 583\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of training examples:\", len(train_dataset))\n",
    "print(\"Number of validation examples:\", len(eval_dataset))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c2b8c84cb8893a0",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "让我们验证训练数据集中的一个例子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "146f8fcc13e43038",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:16:20.873305Z",
     "start_time": "2025-01-09T08:16:20.836610800Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from mindspore.dataset import GeneratorDataset, BatchDataset\n",
    "\n",
    "train_dataloader = GeneratorDataset(train_dataset, column_names=[\"pixel_values\", \"labels\"], shuffle=True).batch(4)\n",
    "eval_dataloader = GeneratorDataset(eval_dataset, column_names=[\"pixel_values\", \"labels\"], shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d034cfe7112511ed",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:16:23.072953200Z",
     "start_time": "2025-01-09T08:16:22.941720800Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3, 384, 384)\n"
     ]
    }
   ],
   "source": [
    "for data in eval_dataloader.create_dict_iterator():\n",
    "    print(data['pixel_values'].shape)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d546d106169ae6c",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "我们还可以检查原始图像并解码标签："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b7431d709adfd85a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:16:24.362138100Z",
     "start_time": "2025-01-09T08:16:24.205081300Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[    0  5632   957     8  3163  2156     5 11834 31520  3811  2156     2\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100  -100\n",
      "  -100  -100  -100  -100  -100  -100  -100  -100]\n",
      "[    0  5632   957     8  3163  2156     5 11834 31520  3811  2156     2\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1     1     1     1     1\n",
      "     1     1     1     1     1     1     1     1]\n",
      "with James and Charles, the Puritans argued,\n"
     ]
    }
   ],
   "source": [
    "image = Image.open(train_dataset.root_dir + '/' + train_df['file_name'][0]).convert(\"RGB\")\n",
    "image.show()\n",
    "labels = train_dataset[0][1]\n",
    "print(labels)\n",
    "labels[labels == -100] = processor.tokenizer.pad_token_id\n",
    "print(labels)\n",
    "label_str = processor.decode(labels, skip_special_tokens=True)\n",
    "print(label_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "388c9d27a1eac60b",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "在这里，我们根据预训练权重初始化 TrOCR 模型。请注意，语言建模头的权重已从预训练中初始化，因为模型已在预训练阶段训练为生成文本。有关详细信息，请参阅论文。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "94b6dfc5a4914634",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:16:40.972904500Z",
     "start_time": "2025-01-09T08:16:28.528055300Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at microsoft/trocr-base-stage1 were not used when initializing VisionEncoderDecoderModel: ['decoder.model.decoder.embed_positions._float_tensor']\n",
      "- This IS expected if you are initializing VisionEncoderDecoderModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing VisionEncoderDecoderModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of VisionEncoderDecoderModel were not initialized from the model checkpoint at microsoft/trocr-base-stage1 and are newly initialized: ['encoder.pooler.dense.bias', 'encoder.pooler.dense.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<mindnlp.transformers.models.vision_encoder_decoder.modeling_vision_encoder_decoder.VisionEncoderDecoderModel object at 0x000002AABC735E20>\n"
     ]
    }
   ],
   "source": [
    "from mindnlp.transformers import VisionEncoderDecoderModel\n",
    "\n",
    "model = VisionEncoderDecoderModel.from_pretrained(\"microsoft/trocr-base-stage1\")\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac5a24c4028cdcf8",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "我们需要设置几个属性，即：\n",
    "- 从标签创建decoder_input_ids所需的属性（模型将通过将标签向右移动一个位置并在前面添加decoder_start_token_id，以及用pad_token_id替换-）\n",
    "- 模型的词汇量（用于解码器顶部的语言建模头）\n",
    "- 生成文本时使用的波束搜索相关参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3f057c3603a6ed23",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:16:55.902312900Z",
     "start_time": "2025-01-09T08:16:55.887700300Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# set special tokens used for creating the decoder_input_ids from the labels\n",
    "model.config.decoder_start_token_id = processor.tokenizer.cls_token_id\n",
    "model.config.pad_token_id = processor.tokenizer.pad_token_id\n",
    "# make sure vocab size is set correctly\n",
    "model.config.vocab_size = model.config.decoder.vocab_size\n",
    "\n",
    "# set beam search parameters\n",
    "model.config.eos_token_id = processor.tokenizer.sep_token_id\n",
    "model.config.max_length = 64\n",
    "model.config.early_stopping = True\n",
    "model.config.no_repeat_ngram_size = 3\n",
    "model.config.length_penalty = 2.0\n",
    "model.config.num_beams = 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "73ec263a86ca946d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:17:00.476507500Z",
     "start_time": "2025-01-09T08:16:56.912798800Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from mindnlp import evaluate\n",
    "\n",
    "cer_metric = evaluate.load(\"cer\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5cdf3ff4406fbe5d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:17:02.218606600Z",
     "start_time": "2025-01-09T08:17:02.197829Z"
    },
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def compute_cer(pred_ids, label_ids):\n",
    "    pred_str = processor.batch_decode(pred_ids, skip_special_tokens=True)\n",
    "    label_ids[label_ids == -100] = processor.tokenizer.pad_token_id\n",
    "    label_str = processor.batch_decode(label_ids, skip_special_tokens=True)\n",
    "\n",
    "    cer = cer_metric.compute(predictions=pred_str, references=label_str)\n",
    "\n",
    "    return cer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e42f03cd1746487",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-01-09T08:25:43.550682500Z",
     "start_time": "2025-01-09T08:17:09.002942800Z"
    },
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "6it [08:26, 84.34s/it]\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[13], line 26\u001b[0m\n\u001b[0;32m     21\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m loss\n\u001b[0;32m     24\u001b[0m grad_fn \u001b[38;5;241m=\u001b[39m mindspore\u001b[38;5;241m.\u001b[39mvalue_and_grad(compute_loss, \u001b[38;5;28;01mNone\u001b[39;00m, weights)\n\u001b[1;32m---> 26\u001b[0m loss, grads \u001b[38;5;241m=\u001b[39m \u001b[43mgrad_fn\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpixel_values\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlabels\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     27\u001b[0m optimizer\u001b[38;5;241m.\u001b[39mstep(grads)\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\composite\\base.py:625\u001b[0m, in \u001b[0;36m_Grad.__call__.<locals>.after_grad\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    624\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mafter_grad\u001b[39m(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 625\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mgrad_\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfn_\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mweights\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\common\\api.py:121\u001b[0m, in \u001b[0;36m_wrap_func.<locals>.wrapper\u001b[1;34m(*arg, **kwargs)\u001b[0m\n\u001b[0;32m    119\u001b[0m \u001b[38;5;129m@wraps\u001b[39m(fn)\n\u001b[0;32m    120\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mwrapper\u001b[39m(\u001b[38;5;241m*\u001b[39marg, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 121\u001b[0m     results \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43marg\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    122\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m _convert_python_data(results)\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\composite\\base.py:600\u001b[0m, in \u001b[0;36m_Grad.__call__.<locals>.after_grad\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    598\u001b[0m \u001b[38;5;129m@_wrap_func\u001b[39m\n\u001b[0;32m    599\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mafter_grad\u001b[39m(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 600\u001b[0m     res \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_pynative_forward_run\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgrad_\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mweights\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    601\u001b[0m     _pynative_executor\u001b[38;5;241m.\u001b[39mgrad(fn, grad_, weights, grad_position, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m    602\u001b[0m     out \u001b[38;5;241m=\u001b[39m _pynative_executor()\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\composite\\base.py:650\u001b[0m, in \u001b[0;36m_Grad._pynative_forward_run\u001b[1;34m(self, fn, grad, weights, args, kwargs)\u001b[0m\n\u001b[0;32m    648\u001b[0m _pynative_executor\u001b[38;5;241m.\u001b[39mset_grad_flag(\u001b[38;5;28;01mTrue\u001b[39;00m)\n\u001b[0;32m    649\u001b[0m _pynative_executor\u001b[38;5;241m.\u001b[39mnew_graph(fn, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mnew_kwargs)\n\u001b[1;32m--> 650\u001b[0m outputs \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mnew_kwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    651\u001b[0m _pynative_executor\u001b[38;5;241m.\u001b[39mend_graph(fn, outputs, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mnew_kwargs)\n\u001b[0;32m    652\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m outputs\n",
      "Cell \u001b[1;32mIn[13], line 19\u001b[0m, in \u001b[0;36mcompute_loss\u001b[1;34m(pixel_values, labels)\u001b[0m\n\u001b[0;32m     18\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcompute_loss\u001b[39m(pixel_values, labels):\n\u001b[1;32m---> 19\u001b[0m     output \u001b[38;5;241m=\u001b[39m \u001b[43mmodel\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpixel_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mpixel_values\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlabels\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlabels\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     20\u001b[0m     loss \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mloss\n\u001b[0;32m     21\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m loss\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:338\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    337\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_wrapped_call_impl\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 338\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:349\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    344\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m    345\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m    347\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m    348\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m--> 349\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    351\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m    352\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\transformers\\models\\vision_encoder_decoder\\modeling_vision_encoder_decoder.py:394\u001b[0m, in \u001b[0;36mVisionEncoderDecoderModel.forward\u001b[1;34m(self, pixel_values, decoder_input_ids, decoder_attention_mask, encoder_outputs, past_key_values, decoder_inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict, **kwargs)\u001b[0m\n\u001b[0;32m    391\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m pixel_values \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m    392\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mYou have to specify pixel_values\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m--> 394\u001b[0m     encoder_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mencoder\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    395\u001b[0m \u001b[43m        \u001b[49m\u001b[43mpixel_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mpixel_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    396\u001b[0m \u001b[43m        \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    397\u001b[0m \u001b[43m        \u001b[49m\u001b[43moutput_hidden_states\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    398\u001b[0m \u001b[43m        \u001b[49m\u001b[43mreturn_dict\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mreturn_dict\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    399\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs_encoder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    400\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    401\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(encoder_outputs, \u001b[38;5;28mtuple\u001b[39m):\n\u001b[0;32m    402\u001b[0m     encoder_outputs \u001b[38;5;241m=\u001b[39m BaseModelOutput(\u001b[38;5;241m*\u001b[39mencoder_outputs)\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:338\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    337\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_wrapped_call_impl\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 338\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:349\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    344\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m    345\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m    347\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m    348\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m--> 349\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    351\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m    352\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\transformers\\models\\vit\\modeling_vit.py:512\u001b[0m, in \u001b[0;36mViTModel.forward\u001b[1;34m(self, pixel_values, bool_masked_pos, head_mask, output_attentions, output_hidden_states, interpolate_pos_encoding, return_dict)\u001b[0m\n\u001b[0;32m    506\u001b[0m     pixel_values \u001b[38;5;241m=\u001b[39m pixel_values\u001b[38;5;241m.\u001b[39mto(expected_dtype)\n\u001b[0;32m    508\u001b[0m embedding_output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39membeddings(\n\u001b[0;32m    509\u001b[0m     pixel_values, bool_masked_pos\u001b[38;5;241m=\u001b[39mbool_masked_pos, interpolate_pos_encoding\u001b[38;5;241m=\u001b[39minterpolate_pos_encoding\n\u001b[0;32m    510\u001b[0m )\n\u001b[1;32m--> 512\u001b[0m encoder_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mencoder\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    513\u001b[0m \u001b[43m    \u001b[49m\u001b[43membedding_output\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    514\u001b[0m \u001b[43m    \u001b[49m\u001b[43mhead_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhead_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    515\u001b[0m \u001b[43m    \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    516\u001b[0m \u001b[43m    \u001b[49m\u001b[43moutput_hidden_states\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    517\u001b[0m \u001b[43m    \u001b[49m\u001b[43mreturn_dict\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mreturn_dict\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    518\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    520\u001b[0m sequence_output \u001b[38;5;241m=\u001b[39m encoder_outputs[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m    521\u001b[0m sequence_output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlayernorm(sequence_output)\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:338\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    337\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_wrapped_call_impl\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 338\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:349\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    344\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m    345\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m    347\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m    348\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m--> 349\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    351\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m    352\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\transformers\\models\\vit\\modeling_vit.py:396\u001b[0m, in \u001b[0;36mViTEncoder.forward\u001b[1;34m(self, hidden_states, head_mask, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[0;32m    389\u001b[0m     layer_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_gradient_checkpointing_func(\n\u001b[0;32m    390\u001b[0m         layer_module\u001b[38;5;241m.\u001b[39m\u001b[38;5;21m__call__\u001b[39m,\n\u001b[0;32m    391\u001b[0m         hidden_states,\n\u001b[0;32m    392\u001b[0m         layer_head_mask,\n\u001b[0;32m    393\u001b[0m         output_attentions,\n\u001b[0;32m    394\u001b[0m     )\n\u001b[0;32m    395\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 396\u001b[0m     layer_outputs \u001b[38;5;241m=\u001b[39m \u001b[43mlayer_module\u001b[49m\u001b[43m(\u001b[49m\u001b[43mhidden_states\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlayer_head_mask\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    398\u001b[0m hidden_states \u001b[38;5;241m=\u001b[39m layer_outputs[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m    400\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m output_attentions:\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:338\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    337\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_wrapped_call_impl\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 338\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:349\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    344\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m    345\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m    347\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m    348\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m--> 349\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    351\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m    352\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\transformers\\models\\vit\\modeling_vit.py:341\u001b[0m, in \u001b[0;36mViTLayer.forward\u001b[1;34m(self, hidden_states, head_mask, output_attentions)\u001b[0m\n\u001b[0;32m    335\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mforward\u001b[39m(\n\u001b[0;32m    336\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m    337\u001b[0m     hidden_states: mindspore\u001b[38;5;241m.\u001b[39mTensor,\n\u001b[0;32m    338\u001b[0m     head_mask: Optional[mindspore\u001b[38;5;241m.\u001b[39mTensor] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m    339\u001b[0m     output_attentions: \u001b[38;5;28mbool\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mFalse\u001b[39;00m,\n\u001b[0;32m    340\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Union[Tuple[mindspore\u001b[38;5;241m.\u001b[39mTensor, mindspore\u001b[38;5;241m.\u001b[39mTensor], Tuple[mindspore\u001b[38;5;241m.\u001b[39mTensor]]:\n\u001b[1;32m--> 341\u001b[0m     self_attention_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mattention\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    342\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mlayernorm_before\u001b[49m\u001b[43m(\u001b[49m\u001b[43mhidden_states\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\u001b[43m  \u001b[49m\u001b[38;5;66;43;03m# in ViT, layernorm is applied before self-attention\u001b[39;49;00m\n\u001b[0;32m    343\u001b[0m \u001b[43m        \u001b[49m\u001b[43mhead_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    344\u001b[0m \u001b[43m        \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m    345\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    346\u001b[0m     attention_output \u001b[38;5;241m=\u001b[39m self_attention_outputs[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m    347\u001b[0m     outputs \u001b[38;5;241m=\u001b[39m self_attention_outputs[\u001b[38;5;241m1\u001b[39m:]  \u001b[38;5;66;03m# add self attentions if we output attention weights\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:338\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    337\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_wrapped_call_impl\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 338\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:349\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    344\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m    345\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m    347\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m    348\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m--> 349\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    351\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m    352\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\transformers\\models\\vit\\modeling_vit.py:283\u001b[0m, in \u001b[0;36mViTAttention.forward\u001b[1;34m(self, hidden_states, head_mask, output_attentions)\u001b[0m\n\u001b[0;32m    277\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mforward\u001b[39m(\n\u001b[0;32m    278\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m    279\u001b[0m     hidden_states: mindspore\u001b[38;5;241m.\u001b[39mTensor,\n\u001b[0;32m    280\u001b[0m     head_mask: Optional[mindspore\u001b[38;5;241m.\u001b[39mTensor] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m    281\u001b[0m     output_attentions: \u001b[38;5;28mbool\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mFalse\u001b[39;00m,\n\u001b[0;32m    282\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Union[Tuple[mindspore\u001b[38;5;241m.\u001b[39mTensor, mindspore\u001b[38;5;241m.\u001b[39mTensor], Tuple[mindspore\u001b[38;5;241m.\u001b[39mTensor]]:\n\u001b[1;32m--> 283\u001b[0m     self_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mattention\u001b[49m\u001b[43m(\u001b[49m\u001b[43mhidden_states\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mhead_mask\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    285\u001b[0m     attention_output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39moutput(self_outputs[\u001b[38;5;241m0\u001b[39m], hidden_states)\n\u001b[0;32m    287\u001b[0m     outputs \u001b[38;5;241m=\u001b[39m (attention_output,) \u001b[38;5;241m+\u001b[39m self_outputs[\u001b[38;5;241m1\u001b[39m:]  \u001b[38;5;66;03m# add attentions if we output them\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:338\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    337\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_wrapped_call_impl\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[1;32m--> 338\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_call_impl\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\nn\\modules\\module.py:349\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m    344\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m    345\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m    346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m    347\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m    348\u001b[0m         \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m--> 349\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mforward_call\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    351\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m    352\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\transformers\\models\\vit\\modeling_vit.py:223\u001b[0m, in \u001b[0;36mViTSelfAttention.forward\u001b[1;34m(self, hidden_states, head_mask, output_attentions)\u001b[0m\n\u001b[0;32m    220\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m head_mask \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m    221\u001b[0m     attention_probs \u001b[38;5;241m=\u001b[39m attention_probs \u001b[38;5;241m*\u001b[39m head_mask\n\u001b[1;32m--> 223\u001b[0m context_layer \u001b[38;5;241m=\u001b[39m \u001b[43mops\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmatmul\u001b[49m\u001b[43m(\u001b[49m\u001b[43mattention_probs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mvalue_layer\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    225\u001b[0m context_layer \u001b[38;5;241m=\u001b[39m context_layer\u001b[38;5;241m.\u001b[39mpermute(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;241m2\u001b[39m, \u001b[38;5;241m1\u001b[39m, \u001b[38;5;241m3\u001b[39m)\n\u001b[0;32m    226\u001b[0m new_context_layer_shape \u001b[38;5;241m=\u001b[39m context_layer\u001b[38;5;241m.\u001b[39mshape[:\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m2\u001b[39m] \u001b[38;5;241m+\u001b[39m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mall_head_size,)\n",
      "File \u001b[1;32m~\\Documents\\GitHub\\mindnlp\\mindnlp\\core\\ops\\blas.py:69\u001b[0m, in \u001b[0;36mmatmul\u001b[1;34m(input, other)\u001b[0m\n\u001b[0;32m     67\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m USE_PYBOOST:\n\u001b[0;32m     68\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m mindspore\u001b[38;5;241m.\u001b[39mmint\u001b[38;5;241m.\u001b[39mmatmul(\u001b[38;5;28minput\u001b[39m, other)\n\u001b[1;32m---> 69\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mops\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmatmul\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mother\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\function\\math_func.py:9691\u001b[0m, in \u001b[0;36mmatmul\u001b[1;34m(input, other)\u001b[0m\n\u001b[0;32m   9689\u001b[0m other \u001b[38;5;241m=\u001b[39m _expand(other, ndim_aligned)\n\u001b[0;32m   9690\u001b[0m shape1_aligned, shape2_aligned \u001b[38;5;241m=\u001b[39m shape_(\u001b[38;5;28minput\u001b[39m), shape_(other)\n\u001b[1;32m-> 9691\u001b[0m \u001b[38;5;28minput\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[43m_broadcast_to\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshape1_aligned\u001b[49m\u001b[43m[\u001b[49m\u001b[43m:\u001b[49m\u001b[38;5;241;43m-\u001b[39;49m\u001b[38;5;241;43m2\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshape_backbone\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mndim_aligned\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m   9692\u001b[0m other \u001b[38;5;241m=\u001b[39m _broadcast_to(other, shape2_aligned[:\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m2\u001b[39m], shape_backbone, ndim_aligned)\n\u001b[0;32m   9693\u001b[0m res \u001b[38;5;241m=\u001b[39m _batch_matmul(\u001b[38;5;28minput\u001b[39m, other)\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\function\\math_func.py:9598\u001b[0m, in \u001b[0;36m_broadcast_to\u001b[1;34m(x, shape_cur, shape_to, ndim_to)\u001b[0m\n\u001b[0;32m   9596\u001b[0m tile_size_op \u001b[38;5;241m=\u001b[39m _get_cache_prim(TileSize)()\n\u001b[0;32m   9597\u001b[0m size \u001b[38;5;241m=\u001b[39m tile_size_op(shape_cur, shape_to, ndim_to)\n\u001b[1;32m-> 9598\u001b[0m \u001b[43mF\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mstop_gradient\u001b[49m\u001b[43m(\u001b[49m\u001b[43msize\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m   9599\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m tile_(x, size)\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\function\\grad\\grad_func.py:1401\u001b[0m, in \u001b[0;36mstop_gradient\u001b[1;34m(value)\u001b[0m\n\u001b[0;32m   1367\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mstop_gradient\u001b[39m(value):\n\u001b[0;32m   1368\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[0;32m   1369\u001b[0m \u001b[38;5;124;03m    StopGradient is used for eliminating the effect of a value on the gradient, such as truncating\u001b[39;00m\n\u001b[0;32m   1370\u001b[0m \u001b[38;5;124;03m    the gradient propagation from an output of a function.\u001b[39;00m\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m   1399\u001b[0m \u001b[38;5;124;03m         [1.4100001 1.6       6.5999994]]\u001b[39;00m\n\u001b[0;32m   1400\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[1;32m-> 1401\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mP\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mStopGradient\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\u001b[43mvalue\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\primitive.py:314\u001b[0m, in \u001b[0;36mPrimitive.__call__\u001b[1;34m(self, *args)\u001b[0m\n\u001b[0;32m    312\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m should_elim:\n\u001b[0;32m    313\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m output\n\u001b[1;32m--> 314\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43m_run_op\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\ops\\primitive.py:913\u001b[0m, in \u001b[0;36m_run_op\u001b[1;34m(obj, op_name, args)\u001b[0m\n\u001b[0;32m    911\u001b[0m         \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(arg, Parameter) \u001b[38;5;129;01mand\u001b[39;00m arg\u001b[38;5;241m.\u001b[39mhas_init:\n\u001b[0;32m    912\u001b[0m             arg\u001b[38;5;241m.\u001b[39minit_data()\n\u001b[1;32m--> 913\u001b[0m     stub \u001b[38;5;241m=\u001b[39m \u001b[43m_pynative_executor\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_op_async\u001b[49m\u001b[43m(\u001b[49m\u001b[43mobj\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mop_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    914\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m _convert_stub(stub)\n\u001b[0;32m    915\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m _RunOpHook\u001b[38;5;241m.\u001b[39mcurrent\u001b[38;5;241m.\u001b[39mhook(obj, args)\n",
      "File \u001b[1;32m~\\.conda\\envs\\mindspore\\lib\\site-packages\\mindspore\\common\\api.py:1186\u001b[0m, in \u001b[0;36m_PyNativeExecutor.run_op_async\u001b[1;34m(self, *args)\u001b[0m\n\u001b[0;32m   1176\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mrun_op_async\u001b[39m(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs):\n\u001b[0;32m   1177\u001b[0m \u001b[38;5;250m    \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[0;32m   1178\u001b[0m \u001b[38;5;124;03m    Run single op async.\u001b[39;00m\n\u001b[0;32m   1179\u001b[0m \n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m   1184\u001b[0m \u001b[38;5;124;03m        StubNode, result of run op.\u001b[39;00m\n\u001b[0;32m   1185\u001b[0m \u001b[38;5;124;03m    \"\"\"\u001b[39;00m\n\u001b[1;32m-> 1186\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_executor\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_op_async\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m)\u001b[49m\n",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "from mindnlp.core.optim import AdamW\n",
    "from tqdm import tqdm\n",
    "import mindspore\n",
    "\n",
    "optimizer = AdamW(model.parameters(), lr=5e-5)\n",
    "for epoch in range(1):\n",
    "    # train\n",
    "    model.set_train()\n",
    "    train_loss = 0.0\n",
    "    for bacth in tqdm(train_dataloader.create_dict_iterator()):\n",
    "        pixel_values = bacth['pixel_values']\n",
    "        labels = bacth['labels']\n",
    "        weights = ()\n",
    "        for group in optimizer.param_groups:\n",
    "            weights += tuple(group['params'])\n",
    "\n",
    "\n",
    "        def compute_loss(pixel_values, labels):\n",
    "            output = model(pixel_values=pixel_values, labels=labels)\n",
    "            loss = output.loss\n",
    "            return loss\n",
    "\n",
    "\n",
    "        grad_fn = mindspore.value_and_grad(compute_loss, None, weights)\n",
    "\n",
    "        loss, grads = grad_fn(pixel_values, labels)\n",
    "        optimizer.step(grads)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d80b5f6b0db98a9",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
